Omnidirectional underwater surveying and telepresence

Exploratory dives are traditionally the first step for marine scientists to acquire information on a previously unknown area of scientific interest. Manned submersibles have been the platform of choice for such exploration, as they allow a high level of environmental perception by the scientist on-board, and the ability to take informed decisions on what to explore next. However, manned submersibles have extremely high operation costs and provide very limited bottom time. Remotely operated vehicles (ROVs) can partially address these two issues, but have operational and cost constraints that restrict their usage. This paper discusses new capabilities to assist scientists operating lightweight hybrid remotely operated vehicles (HROV) in exploratory missions of mapping and surveying. The new capabilities, under development within the Spanish National project OMNIUS, provide a new layer of autonomy for HROVs by exploring three key concepts: Omni-directional optical sensing for collaborative immersive exploration, Proximity safety awareness and Online mapping during mission time.Peer Reviewe

Underwater Gigamosaicing

El procés de fusió de dues o més imatges de la mateixa escena en una d'única i més gran és conegut com a Image Mosaicing. Un cop finalitzat el procés de construcció d'un mosaic, els límits entre les imatges són habitualment visibles, degut a imprecisions en els registres fotomètric i geomètric. L'Image Blending és l'etapa del procediment de mosaicing a la que aquests artefactes són minimitzats o suprimits. Existeixen diverses metodologies a la literatura que tracten aquests problemes, però la majoria es troben orientades a la creació de panorames terrestres, imatges artístiques d'alta resolució o altres aplicacions a les quals el posicionament de la càmera o l'adquisició de les imatges no són etapes rellevants. El treball amb imatges subaquàtiques presenta desafiaments importants, degut a la presència d'scattering (reflexions de partícules en suspensió) i atenuació de la llum i a condicions físiques extremes a milers de metres de profunditat, amb control limitat dels sistemes d'adquisició i la utilització de tecnologia d'alt cost. Imatges amb il·luminació artificial similar, sense llum global com la oferta pel sol, han de ser unides sense mostrar una unió perceptible. Les imatges adquirides a gran profunditat presenten una qualitat altament depenent de la profunditat, i la seva degradació amb aquest factor és molt rellevant. El principal objectiu del treball és presentar dels principals problemes de la imatge subaquàtica, seleccionar les estratègies més adequades i tractar tota la seqüència adquisició-procesament-visualització del procés. Els resultats obtinguts demostren que la solució desenvolupada, basada en una Estratègia de Selecció de Límit Òptim, Fusió en el Domini del Gradient a les regions comunes i Emfatització Adaptativa d'Imatges amb baix nivell de detall permet obtenir uns resultats amb una alta qualitat. També s'ha proposat una estratègia, amb possibilitat d'implementació paral·lela, que permet processar mosaics de kilòmetres d'extensió amb resolució de centímetres per píxel.The process of fusing two or more images of the same scene into a single and larger one is known as a Image Mosaicing. After the mosaic building process, boundary seams along the common overlapping regions are often noticeable, due to photometrical and geometrical registration inaccuracies. Image blending is the step of the mosaicing procedure where those artifacts are minimized or totally suppressed, when it is possible. There are several methodologies in the literature focused on dealing with those problems, but most of them are oriented to the creation of aerial or terrestrial panoramas, high resolution artwork reproductions, and other applications where camera placement and image acquisition are not especially relevant steps of the procedure. Underwater imaging presents special challenges, due to scattering and light attenuation, and to the extreme physical conditions under some thousands of meters of water depth, with limited remote control, using very expensive technology. Images with similar local artificial lighting, without a global illumination, like the one provided by the sun in terrestrial imaging, should be unified in a seamless manner. Furthermore, deep water images present depthdepending highly different quality, increasing the complexity of the seamless merging. The main aim of the work is to present the special underwater imaging problems, give solution by adequate selecting the strategies and deal with the whole acquisition-computation-display pipeline. The obtained results demonstrate that the developed approach, based on Optimal Seam Finding strategy, Intelligent Gradient Domain Blending in the common overlapping regions and Adaptive Image Enhancement for regions with low detail, allow to reach high quality level results. A Large Scale Mosaic Blending methodology, allowing to use parallel processing techniques, has also been proposed, in order process extremely large scale mosaics, of several square kilometers, with very high resolution, of some centimeters by pixel

Image blending techniques and their application in underwater mosaicing

The fusion of several images of the same scene into a single and larger composite is known as photo-mosaic. Unfortunately, the seams along image boundaries are often noticeable, due to photometrical and geometrical registration inaccuracies. Image blending is the merging step in which those artefacts are minimized. Processing bottlenecks and the lack of medium-specific processing tools have restricted underwater photo-mosaics to small areas despite the hundreds of thousands of square meters that modern surveys can cover. Producing these mosaics is difficult due to the challenging nature of the underwater environment and the image acquisition conditions. This thesis proposes strategies and solutions to tackle the problems of very large underwater optical surveys (Giga-mosaics), presenting contributions in the image preprocessing, enhancing and blending steps, resulting in an improved visual quality in the final photo-mosaicLa unió de diverses imatges d’una mateixa escena en una d’única i més gran és coneguda com a foto-mosaic. Malauradament, els límits de les imatges són habitualment perceptibles, degut a imprecisions en els registres fotomètric i geomètric. La fusió d'imatges és l'etapa del procés d'unió a la qual aquests artefactes són minimitzats. Els colls d'ampolla en el processament i la manca d'eines específiques pel tractament del medi han restringint els foto-mosaics submarins a àrees reduïdes, malgrat que els estudis actuals poden cobrir centenars de milers de m2. . La producció d'aquests mosaics és complexa donada la naturalesa del medi subaquàtic i les condicions d'adquisició de les imatges. Aquesta tesi proposa estratègies i solucions per afrontar el problema de la generació de foto-mosaics submarins de grans dimensions (Giga-mosaics), i presenta contribucions en les etapes de preprocessament, realçat i fusió d’imatges, donant lloc a una qualitat visual millorada del foto-mosaic fina

Planar homography: accuracy analysis and applications

Projective homography sits at the heart of many problems in image registration. In addition to many methods for estimating the homography parameters (R.I. Hartley and A. Zisserman, 2000), analytical expressions to assess the accuracy of the transformation parameters have been proposed (A. Criminisi et al., 1999). We show that these expressions provide less accurate bounds than those based on the earlier results of Weng et al. (1989). The discrepancy becomes more critical in applications involving the integration of frame-to-frame homographies and their uncertainties, as in the reconstruction of terrain mosaics and the camera trajectory from flyover imagery. We demonstrate these issues through selected example

Planar homography: accuracy analysis and applications

Projective homography sits at the heart of many problems in image registration. In addition to many methods for estimating the homography parameters (R.I. Hartley and A. Zisserman, 2000), analytical expressions to assess the accuracy of the transformation parameters have been proposed (A. Criminisi et al., 1999). We show that these expressions provide less accurate bounds than those based on the earlier results of Weng et al. (1989). The discrepancy becomes more critical in applications involving the integration of frame-to-frame homographies and their uncertainties, as in the reconstruction of terrain mosaics and the camera trajectory from flyover imagery. We demonstrate these issues through selected example

Challenges of Close-Range Underwater Optical Mapping

Underwater optical mapping often involves the use of image mosaicing techniques. High quality mosaicing requires the application of blending methods to achieve continuous and artifact-free mosaics. Image blending has a dilated history of over three decades in the terrestrial and aerial fields. Unfortunately, the nature of the underwater medium adds additional difficulties to the mosaicing and blending tasks. In this paper a survey of the blending methods is given, focusing the attention on its applicability to underwater mosaicing. Image acquisition is performed by Autonomous Underwater Vehicles (AUVs) or Remotely Operated Vehicles (ROVs) in the deep ocean, a medium with aggressive light absorption and disrupting scattering effects that requires of the use of artificial lighting. A comprehensive comparison of the basic features and limitations of some of the most important existing blending techniques is presented. The goal is the generation of seamless and visually pleasant large area photo-mosaics of the seafloor, free from double contouring, ghosting and other disturbing and common blending artifact

A Motion compensated filtering approach to remove sunlight flicker in shallow water images

A common problem in video surveys in very shallow waters is the presence of strong light fluctuations, due to sun light refraction. Refracted sunlight casts fast moving patterns, which can significantly degrade the quality of the acquired data. Motivated by the growing need to improve the quality of shallow water imagery, we propose a method to remove sunlight patterns in video sequences. The method exploits the fact that video sequences allow several observations of the same area of the sea floor, over time. It is based on computing the image difference between a given reference frame and the temporal median of a registered set of neighboring images. A key observation is that this difference will have two components with separable spectral content. One is related to the illumination field (lower spatial frequencies) and the other to the registration error (higher frequencies). The illumination field, recovered by lowpass filtering, is used to correct the reference image. In addition to removing the sunflickering patterns, an important advantage of the approach is the ability to preserve the sharpness in corrected image, even in the presence of registration inaccuracies. The effectiveness of the method is illustrated in image sets acquired under strong camera motion containing non-rigid benthic structures. The results testify the good performance and generality of the approac

Structure, temporal evolution, and heat flux estimates from the Lucky Strike deep-sea hydrothermal field derived from seafloor image mosaics

Here we demonstrate with a study of the Lucky Strike hydrothermal field that image mosaicing over large seafloor areas is feasible with new image processing techniques, and that repeated surveys allow temporal studies of active processes. Lucky Strike mosaics, generated from >56,000 images acquired in 1996, 2006, 2008 and 2009, reveal the distribution and types of diffuse outflow throughout the field, and their association with high-temperature vents. In detail, the zones of outflow are largely controlled by faults, and we suggest that the spatial clustering of active zones likely reflects the geometry of the underlying plumbing system. Imagery also provides constraints on temporal variability at two time-scales. First, based upon changes in individual outflow features identified in mosaics acquired in different years, we document a general decline of diffuse outflow throughout the vent field over time-scales up to 13 years. Second, the image mosaics reveal broad patches of seafloor that we interpret as fossil outflow zones, owing to their association with extinct chimneys and hydrothermal deposits. These areas encompass the entire region of present-day hydrothermal activity, suggesting that the plumbing system has persisted over long periods of time, loosely constrained to hundreds to thousands of years. The coupling of mosaic interpretation and available field measurements allow us to independently estimate the heat flux of the Lucky Strike system at 200 to 1000 MW, with 75% to >90% of this flux taken up by diffuse hydrothermal outflow. Based on these heat flux estimates, we propose that the temporal decline of the system at short and long time scales may be explained by the progressive cooling of the AMC, without replenishment. The results at Lucky Strike demonstrate that repeated image surveys can be routinely performed to characterize and study the temporal variability of a broad range of vent sites hosting active processes (e.g., cold seeps, hydrothermal fields, gas outflows, etc.), allowing a better understanding of fluid flow dynamics from the sub-seafloor, and a quantification of fluxesThis project was funded by CNRS/IFREMER through the 2006, 2008, 2009 and 2010 cruises within the MoMAR program (France), by ANR (France) Mothseim Project NT05-3 42213 to J. Escartín, and by grant CTM2010-15216/MAR from the Spanish Ministry of Science to R. Garcia and J. Escartín. T. Barreyre was supported by University Paris Diderot (Paris 7–France) and Institut de Physique du Globe de Paris (IPGP, France). E. Mittelstaedt was supported by the International Research Fellowship Program of the U.S. National Science Foundation (OISE-0757920

A Novel blending technique for underwater gigamosaicing

The fusion of several images of the same scene into a single and larger composite is known as photomosaicing. Unfortunately, the seams along image boundaries are often noticeable, due to photometrical and geometrical registration inaccuracies. Image blending is the merging step in which those artifacts are minimized. Processing bottlenecks and the lack of medium-specific processing tools have restricted underwater photomosaics to small areas despite the hundreds of thousands of square meters that modern surveys can cover. Large underwater photomosaics are increasingly in demand for the characterization of the seafloor for scientific purposes. Producing these mosaics is difficult due to the challenging nature of the underwater environment and the image acquisition conditions, including extreme depth, scattering and light attenuation phenomena, and difficulties in vehicle navigation and positioning. This paper proposes strategies and solutions to tackle the problems of very large underwater optical surveys (gigamosaics), presenting contributions in the image preprocessing, enhancing, and blending steps, resulting in an improved visual quality in the final photomosaic. A comprehensive review of the existing methods is also presented and discussed. Our approach is validated by a large optical survey of a deep-sea hydrothermal field, leading to a high-quality composite in excess of 5 Gpixe

Feature extraction for underwater visual SLAM

Detecting and selecting proper landmarks is a key issue to solve Simultaneous Localization and Mapping (SLAM). In this work, we present a novel approach to perform this landmark detection. Our approach is based on using three sources of information: 1) three-dimensional topological information from SLAM; 2) context information to characterize regions of interest (RoI); and 3) features extracted from these RoIs. Topological information is taken from the SLAM algorithm, i.e. the three-dimensional approximate position of the landmark with a certain level of uncertainty. Contextual information is obtained by segmenting the image into background and RoIs. Features extracted from points of interest are then computed by using common feature extractors such as SIFT and SURF. This information is used to associate new observations with known landmarks obtained from previous observations. The proposed approach is tested under a real unstructured underwater environment using the SPARUS AUV. Results demonstrate the validity of our approach, improving map consistencyThis work was partially funded through the Spanish Ministry of Education and Science (MCINN) under grant CTM2010-15216 and the EU under grant FP7-ICT-2009-24849